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ABSTRACT 

This paper reports on research that was designed to 
produce a limited number of assessment standards judged by exp^^rtc as 
the most important standards in the practice of quality assessment in 
undergraduate higher education. The study collected the opinions of a 
group of national assessment experts and employed a modification of 
the Delphi technique in order to arrive at a group consensus. The 
research resulted in identification of 40 standards that were agreed 
upon by the expert group. The rationale for each of these standards 
is presented. The study's research phases included: (1) the 
clarification of the component of the institution/program to be 
assessed and its context; (2) the designing of the assessment 
process; (3) collecting and analyzing the data; (4) communicating the 
assessment's findings; and (5) using the findings to make 
recommendations, make decisions about improvements, and make 
judgements about quality. The appendix contains descriptions of the 
40 standards. Contains 19 references. (GLR) 
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« The current concept of assessment in higher education rose to the forefront in 

the mid-1980s through national attention on accountability and quality in higher 
education. The public expressed uncertainty about the worth of higher education and 
asked institutions, primarily public institutions, to demonstrate their educational 
effectiveness. These concerns appeared in several national reports, including those 
of the National Institute of Education [14], the Association of American Colleges 
[1], the Southern Region Education Board [16], and the Education Commission of the 
States [6]. 

Quality, accountability, and assessment are not new issues in higher 

education, although each has acquired a slightly different meaning in recent years. 

A traditional meaning of assessment focused on gathering information about an 

individual, usually through multiple tests and procedures, for purposes of placement 

or guidance [10] . In contrast, the term assessment currently has a broader meaning 

that generally applies to the gathering of information to indicate the extent to 

which an institution is achieving what it purports to do, that for which it is 

accountable. This research used the more inclusive definition below. 

Assessment: The use of various methods to gather both 
quantitative and qualitative information at the level of 
program, institution, and/or system, to describe and 
sometimes to make judgments about the inputs, resources, 
and/or outcomes of an undergraduate education for purposes 
of inqprovement and/or accountability (individual student 
diagnostic assessment has been omitted from the context) 
[modification of a definition provided by Boyer and Ewell 
[2]]. 

This usage reflects the expansion of the term assessment, one that has moved 
beyond gathering chiefly quantitative, testing information on the level of 
individual students for diagnostic purposes. Assessment as defined above extends 
the type of information to include gathering both qualitative and quantitative 
information, expands the level of analysis to several possible levels, and includes 
the conduct of assessment for improvement and/or accountability purposes. 

Incorporated in the definition of assessment used in this research, was the 
use of the gathered information to make value judgments about quality. Whether or 
not to include the stage of making judgments in the definition of a^essment is an 
issue of debate in the field [3]. When included, assessment becomes almost 
indistinguishable from evaluation in which values are placed on the object of the 
gathered information. 

The above definition of assessment also illustrates the recent shift of the 
basis for accountability and judgments about quality from an emphasis on efficiency 
to one on effectiveness. Declining enrollments and resources, and increasing costs 
in the 1970s found researchers involved in assessment of efficiency, i.e., a form of 
accountability based on judgments of financial and other efficiency criteria, e.g., 
enrollment patterns, student -faculty ratios, credit hours, and cost per credit hour. 
In contrast, the current bases for accountability and quality judgments are related 
to effectiveness, i.e., the extent to which an institution, program, or student is 
reaching educational goals, goals for which the institution is accountable [9, 12]. 
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BACKGROUND 

National surveys have indicated that a majority of the states have formal 
initiatives labeled assessment, and that a wide variety of assessment progreuns are 
used by institutions [8]. Terenzini [17] provided a useful framework for 
classifying assessment programs based on the purpose and the unit of analysis: 
individual assessment for placement or guidance; individual assessment for gate 
keeping purposes (e.g., rising junior exams, comprehensive exams, and certification 
exeuns); group assessment for purposes of program enhancement; and group assessment 
for ceunpus and program evaluation (e.g., alumni surveys and value added studies). 
This variety of assessment programs appropriately reflects the diversity among 
institutions in their misaiion, the type of entering student, the areas of expertise 
and priorities of the faculty, available resources, teaching and learning progreuns, 
the institutional reward system, and external and internal pressures for assessment. 
Personnel in individual institutions are faced with a wide variety of examples of 
assessment programs, and with what might seem a formidable task of deciding what to 
assess and how to assess at their unique institution. 

NEED FOR RESEARCH 

Much of the assessment literature addresses assessment from a very general 
perspective or a narrow approach such as institution-specif ic information that may 
not be highly transferable to other institutions. Missing from the assessment 
literature are general assessment principles that could be applied to any 
institution regardless of context [19]. Such tools are needed to guide 
practitioners in addressing the question of how to assess, and to encourage the 
conduct of sound assessment programs. 

Evaluators working in higher education have expressed concern about the 
continued lack of specific methodology, models, or theoretical perspectives to 
inform the practice of assessment, despite the applicability of the theory and 
methodology of evaluation to the Assessment enterprise (4, 15]. Although an 
adaptation of Stake's Countenance Evaluation Model was suggested as a freunework for 
the assessment process [18], minimal use has been made in assessment of the 
developed tools used in the field of evaluation that are directly applicable, and 
potentially valuable in the conduct of assessment. 

Evaluation standards are another potentially valuable evaluation resource for 
guiding the conduct of assessment. Within the field of evaluation, sets of 
standards from the Joint Comnittee [11] and the Evaluation Research Society [7] have 
provided guidelines for conducting evaluations. This research was based on the 
assumption that similar standards could provide guidance for the practice of sound 
assessment. The goal of this research was to promote the conduct of high quality 
assessment programs through identification of appropriate assessment standards. 

Using the existing evaluation standards as a basis, the research was designed 
to produce a limited number of assesstrent standards that are judged by experts as 
the most important standards in the practice of quality assessment in undergraduate 
higher education. A limited number of standards for conducting an assessment might 
be more acceptable, and more readily used than the comprehensive sets of standards 
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for conducting general evaluations that are currently available. To improve the 
uj:ility of the set of assessment standards ^ the research was also designed to 
generate an acconqpanying rationale for the inqportance of the standards to lend 
insight into the use of the standards and the general practice of quality 
assessment. 

METHODOLOGY 

A modification of the Delphi technique, a method for arriving at group 
consensus, was used in the study. National assessment experts served as the Delphi 
participants in this research designed to identify standards for the conduct of 
quality assessment progreons. 
Delphi Technicme 

Several factors contributed to the selection of the Delphi from among 
available group methods for reaching consensus* Because a set of assessment 
standards should be applicable in the many different contexts in which assessment 
takes place, obtaining a group of participants reflective of those contexts was 
particularly important to the research. The mail methodology of the Delphi 
facilitates diverse representation by allowing participation of experts who are 
widely geographically disfpersed and whose schedules might not allow group meetings 
[5, 13]. The Delphi methodology also avoids face-to-*face meetings that provide 
response anonymity for the participants. In this research, such anonymity prevented 
any possible effects on participant responses due to the professional prominence or 
assessment approach identified with E)articular experts or institutions. 

Central to a Delphi is a series of succeeding questionnaires that ask for 
opinions or judgements, allow participants to review a summary of the group's 
responses from the previous round, and encourage revision of opinions or judgments. 
This process works toward the goal of a convergence of ideas, i^e., a consensus 
about an issue. The iterative process of questionnaire distribution, analysis, and 
feedback resulted in three rounds in this research. 

First round. A typical Delphi first round for this research would have asked 
participants for a listing of standards i:hey thought were important. A compiled 
list of these standards would have provided an inventory of potentially applicable 
standards. Developing such an inventory was not necessary' in the first round in 
this research, however, because of existing sets of evaluation standards developed 
by groups of professional organizations, i.e.. The Joint Committee [11] and the 
Evaluation Research Society [7] . An initial inventory of 38 standards was compiled 
by the researcher from the two existing sets of evaluation standards after 
eliminating duplicates. The wording of the standards was revised for the context of 
quality assessment in higher ecucation. To assure that the inventory was 
comprehensive, participants were requested to add any standards that they deemed to 
be missing. Participants in two partial pilots added two additional standards to 
the initial compiled set of 38 standards producing the 40 assessment standards used 
in the final research. 

In the first round, participants were asked to rate the importance of the 
standards using a rating scale of one to five (l=very important, 5=not important). 
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Participants rated the inqportance o£ each of the 40 standards in each of the five 

phases of an assessment as defined in the research (a total of 200 ratings) . The 

five phases c£ assessment as outlined for participants included the following: 

Clarifying the Assessment Object and Context: Describing the component 
of institution/program to be assessed and its context, clarifying 
the purpose of assessment i deciding whether to conduct assessment. 

Designing the Assessment: Identifying the questions to be answered, 

developing overall assessment plan including approach to be used 
and tactics for gathering informationi and developing a management 
plan for use of resources. 

Collecting and Analyzing the Data: Collecting and preparing data, 
analyzing and interpreting data. 

Communicating the Findings: Reporting and communicating the collected 
informationi the findings. 

Using the Findings: Using the findings in applying results^ making 
decisions about improvements , making and reporting 
recommendations I and possibly making judgments about quality. 

Second round . In the second-round questionnaire! participants were asked to 

complete two tasks: re-rating and providing rationale statements for their ratings. 

Participants re-rated the importance of the standards that meet the established 

criteria of importance in the first round ratings. Two criteria were used 

throughout the research to define the levels of importance of the standards. The 

criteria were the values of the first quartile (Qi) and the third quartile (Q^) in 

the distribution of responses. For purposes of this research, the standards were 

defined as having the following level of importance based on the value of the 

quartile ratings: 

"very important standard' 

Q^sLO, QjSil.Oi i.e., at least 75% assigned the standard a "1* 
rating of very important, 

"generally important standard" 

Qisl.O, Q|«2.0, i.e., at least 25% of the participants assigned 
the standard a "1" rating of very important and at least an 
additional 50% assigned the standard at least a '2* rating of 
generally important, 

"somewhat inqportant standard" 

Qi»1.0, Qjs3.0, i.e., at least 25% of the participants assigned 
the stamdard a "1* rating of very important and at least an 
additional 50% assigned the standard at least a "3" rating of 
somewhat important. 

In addition to conqplecing inqportance ratings in the second round, participants also 
selected two standards in each phase for which to provide a short rationale of the 
importance of the standards. 

Third round . In the third round participants were asked to reconsider their 
rationale for the importance of the standards. As the basis for that task, a 
summary of the rationale statements from the second round was compiled oy the 
researcher. The compilation work was validated by a monitor team consisting of 
three members who had been active in assessment nationally: a faculty member # an 
administrator, and an executive staff member of a national educational association. 
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The third**round participants read the summarized rationale statements for each 
phase, indicated any gaps they found in the statements of rationale for the 
importance of the standards, and responded to any statements with which they 
disagreed by crossing out the statement and entering their own position. The 
researcher used the responses to revise the summary; revisions were subsequently 
validated by the monitors. 
Expert Participants 

A sample of 50 participants from each of two categories of experts constituted 
the initial total s«unple of 100 assessment experts: 50 individuals with major 
responsibilities in institutions with established, active assessment programs, and 
50 authors of assessment conference papers or articles in journals and publications 
of six professional organizations that had addressed the topic of quality 
assessment.^ No duplication of institutions existed among the participants. A 
total of 85 assessment experts (45 active in assessment and 40 authors) agreed to 
participate. 

In the first-round, a return rate of 82% was achieved with 70 of the 85 
participants returning the questionnaire that requested only ratings. Of the 70 
participants who returned the first round ratings, 53 (76%) returned the second 
round re-ratings and written rationale statements, and 38 (54%) completed the third 
round revisions of their rationale statements. 

RESULTS 

The results of this research included both a set of standards agreed upon by a 
sample of assessment experts as being the most inqportant in the conduct of quality 
assessment programs in each of the five phases of an assessment, and a rationale for 
the importance of each of those standards. The former, i.e., the set of assessment 
standards produced by the importance ratings, is the primary focus of this paper; 
only a few rationale comments will be reported to illuminate the ratings. 
Important standards by Phase 

The number of assessment standards rated as important within each of the five 
phases of assessment used in this research is presented in Table 1. 



Place Table 1 about here 



The research identified 12 standards as being very important, the highest of the 
importance ratings, in at least one phase of an assessment (one standard was rated 
as very important in both of the first two phases) . The number of standards 
identified as being very important differed greatly eunong the assessment phases I 
through IV. As indicated in Table 1, none of the standards were identified as very 
iinportant in Phase V. The inqportance ratings given each standard by assessment 
phase is reported in Table 2. 



Place Table 2 about here 
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Only those assessment standards (AS) meeting the criterion established for the level 
of very important (the highest of the three importance categories) are discussed 
below. 

Phase I: Clarifying Object and Context , The firs phase of the assessment as 

defined in this research included the following tasks: describing the component of 

the institution/program to be assessed and its context, clarifying the purpose of 

the assessment, and deciding whether to conduct the assessment. As indicated in 

Table 2, two standards met the criterion for the highest level of importance, very 

important. Those standards were as follows: 

AS a Deeerlbed Purpoeee end Procedures i The purpose of the 
assessment should be clarified; and procedures 
selected should be appropriate for the purpose. 
Descriptions of the purpose, procedures, and range of 
activities should be in enough detail, so that they 
can be identified and assessed. 

AS 25 Object Zdentl fleet loot The purposes and 

characteristics of the object of the assessment to be 
addressed in the assessment should be specified as 
precisely as possible, e.g., program objectives or 
institutional mission statement. 

Participants indicated in the rationale statements that they viewed the 
clarification of what will be assessed and why to be the major tasks of the first 
phase. Standards AS 2 Described Purposes and Procedures, and AS 25 Object 
Identification were perceived by participants to directly address those tasks and to 
essentially define the first phase. 

Participants noted that an assessment program may be for the purpose of 
improvement in an area such as teaching or learning, or for the purpose of 
accountability to a group such as trustees or legislators, or some combination of 
both. Because assessment for one purpose may not fulfill the needs for another, 
clarifying the purpose was deemed essential in providing appropriate direction and 
guidance for subsequent decisions about the object of the assessment, the 
procedures, and the appropriate tools and strategies. A major reason cited in the 
rationale summary for the inqportance of attending to clarification of the purpose 
related to the consequences of an unclear purpose: an assessment progreun that can 
easily stray from the original intent and result in expenditure of funds but 
unusable information. 

Standard AS 25 Object Identification was /iewed by participants as important 
for reaching an agreement on what will not, ar well as on what will be assessed, and 
for other necessary decisions about subsequent assessment activity. A possible 
outcome identified by participants of an unfocused assessment was a snowball growth 
of assessment activity through attempts to assess everything. 

Phase II: Designing the Assessment . As defined in the research. Phase II 
included the tasks of identifying the questions to be answered, developing an 
overall assessment plan including an approach to be used and tactics for gathering 
information, and developing a management plan for the resources (see Table 2 for 
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standards considered important in Phase II) . The standards rated by participants as 

very important in the second phase included the following: 

AS a Deeorlbed Purpoeee end Proeedureet The purpose of the 
assessment should be claiified; and procedures 
selected should be appropriate for the purpose. 
Descriptions of the purpose i procedures, and range of 
activities should be in enough detail, so that they 
can be identified and assessed. 

AS 33 Rights of Huaan Subjects s Assessments should be 
designed and conducted to protect and respect the 
rights and welfare of the human subjects. 



AS 21 Reliable Measurement t The information-gathering 
instruments and procedures should be chosen or 
developed and then implemented in ways that will 
assure that the information obtained is sufficiently 
reliable for the intended use. 



AS 28 Valid Measureaent t The information-gathering 
instruments anc procedures should be chosen or 
developed and then implemented in ways that will 
assure that the interpretation arrived at is valid for 
the given use» 

The rationale given by participants for assigning high importance of standard 
AS 2 Described Purposes and Procedures in both the first two phases, related to the 
close tie between clarifying the purpose of an assessment in Phase I, and selecting 
an appropriate ass:;essment design for that purpose in Phase II. 

Standard AS 23 Rights of Human Subjects was viewed important in protecting the 
rights of assessment participants. Such protection was deemed essential from an 
ethical, professional, and legal standpoint. In addressir.^* AS 21 Reliable 
Measurement, and AS 28 Valid Measurement in the rationale summary, participants 
acknowledged that reliability and validity of instruments and procedures to gather 
the information are basic in any field of inquiry, and must be considered in 
designing an assessment if the results are to be accepted and of utility. 

Phase III; Collecting and Analyzing Data . This phase was defined for the 
participants as collecting and preparing data, and analyzing and interpreting the 
data (see Table 2 for inqportant standards in Phase III) . The standards identified 
by the participants as those that were very important in the phase of collecting and 
analyzing data included the following: 

AS 9 Iraluator Credibility t The persons conducting the 

assessment should be both trustworthy and competent to 
perform the assessment. 

AS 24 Analysis of Quutltatlve Informations Quantitative 
information in an assessment should be appropriately 
and syst^tmatically analyzed to ensure supportable 
interpretations. Report potential weaknesses in the 
data collection and describe possible influences on 
findings. 

In the rationale statements, participants indicated that they considered 
assessment to be not only a scientific endeavor but also a social, political^ and 
educational activity. Consequently, participants emphasized the importance of the 
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persons conducting ths assassmsnt bsing both compstsnt for the technical v'tndidavors, 
and trustworthy in performing a task so heavily social and political in nature. 
Participants viewed this standard as not only important for the quality of the 
information but also for the credibility of the program. 

The standard AS 24 Analysis of Quantitative Information was the only other 
assessment standard identified as vexy important in this phase by the experts. 
Participants indicated in the rationale summary that such adherence to standards 
regarding the analysis of information was basic to good research as well as sound 
assessment . 

Phase IV; Cowonuni eating the Findings > For the purposes of this research, 
Phase IV was defined as reporting and communicating the collected information^ i.e., 
the findings (see Table 2 for standards identified as important in Phase IV) . 
Participants identified the following five standards as vexy important in reporting 
the findings. 

AS 30 Balanced Reporting of Findings i The assessment should 
be complete and fair in its presentation of the 
findings, e.g., balanced reporting of data that may 
lead to conclusions about strengths as well as 
weaknesses, and inclusion of intended as well a 
unintended outcomes. 

AS 35 Objective Reporting of Results i The assessment 

procedure should provide safeguards to protect the 
assessment findings and reports against distortion by 
the personal feelings and biases of any party to the 
assessment. When appropriate, draft reports should be 
checked with representatives of audiences for clarity 
and accuracy. 

AS 6 Pull and Prank Disclosure t Oral and written material 
containing gathered infoinnation should be open, 
direct, and honest in the disclosure of pertinent 
findings, including the limitations of the assessment. 

AS 34 Report Clarltyi Reports of results should describe 
the object being assessed and its context, and the 
purposes, procedures, and findings of the assessment. 
Limitations caused by constraints on time, resources, 
and data availability should also be stated. 

AS 7 Targeted Disseminations Persons i groups, and 

organizations who have contributed to the assessment 
should receive feedback appropriate to their needs. 
When appropriate, check draft reports with 
representatives of audiences for clarity and accuracy; 
consider tailoring dissemination for specific 
audiences using a variety of formats and approaches. 

Three of the standards identified by participants as very important in this 
phase addressed the content of the report and included AS 35 Objective Reporting of 
Results, AS 20 Balanced Reporting of Findings, and AS 34 Report Clarity. In their 
statements of rationale, participants indicated that objective and balanced 
reporting was critical in presenting a complete picture that can enhance a full, 
broad perspective of the object, and can improve the quality of subsequent 
conclusions and recommendations. 
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• Assessment standards AS 6 Pull and Frank Disclosure, and AS 7 Targeted 
Dissemination, which focused on the dissemination of the report, were also deemed 
very important by participants in this phase. Participants considered these 
standards as important to assure that those needing access to the information would 
obtain it in a form that would enable them to act with full knowledge. This 
rationale was closely linked by participants in the summary to that given for 
Standard AS 7 Targeted Dissemination in which participants acknowledged that 
different audiancefi have different needs for assessment, and that although some data 
may not be appropriate for one audience, the same data ma/ be imperative for the 
work of others* 

Phaje V; Using the Findings . The last phase of an assessment was defined in 
this research to include using the findings in applying results, making decisions 
about improvements, making and reporting recommendations, and possibly making 
judgments about quality. As indicated in Table 2, however, none of the 40 standards 
listed in the original questionnaire inventory were rated by participants as very 
iinportant (the highest category of importance) in the last phase of an assessment. 
Standards Important Across Phases 

An additional analysis was conducted to determine if any standards played a 
consistently important role in all or most phases without necessarily receiving the 
highest importance designation, ver}.' important, in any one phase. Seven of the 
total of 40 standards met the criterion for soinewhat, gmnmrally, or vexy important 
in at least four of the five phases of assessment. The standards were AS 1 Human 
Interactions (ixi^ortant in all five phases), AS 2 Described Purposes and Procedures^ 
AS 9 Evaluator Credibility, AS 23 Rights of Human Subjects, AS 30 Conflict of 
Interest. AS 31 Audience Identification, and AS 32 Unit of Analysis. Four of these 
standards (AS 1, AS 30, AS 31, and AS 32) had not been identified as very important 
(the highest in^rtarce category) in any phase. 

Five of the seven standards that were consistently important across the 
assessmeac phases addressed dimensions with human components. These five standards 
(AS 1, AS 9, AS 23, AS 30, and AS 31) appeared to include all but two of the 
standards in the inventory (AS 12 Audience Cooperation and AS 26 Public's Right-to- 
Know) that addressed issues directly involving individuals. 
Final Set of Assessment Standards 

The research identified a set of standards that were judged by assessment 
experts to be the most important in the conduct of quality assessment programs in 
higher education. The set of 16 assessment standards, identified in Table 2« is 
composed of the 12 standards that received the highest ratings in at least one phase 
and the four standa^-^s that were consistently across the phases rated important to 
some degree. 



The importance ratings and the rationale statements provided in this research 
by the assessment experts identified several areas that were deemed critical but 
which had been identified in an earlier review [19] as receiving sparse attention in 
the assessment literature. A gap between the importance and the available guidance 
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was identified for several assessment dimensions: human and political issues 
including political viability, the rights of human subjects, ethics, conflict of 
interest, and human interactions; collection of information involving the role and 
analysis of qualitative information, and the credibility and competency of persons 
conducting assessment; report writing e^nd dissemination issues of full and frank 
disclosure, right -to-kncw audiences, and targeted reports; and utilization issues 
such aa assessment side-effects, responsibility for use of findings, and promotion 
of utilization. 

To improve the quality of the assessment information available to client 
practitioners, appropriate professional organizations should expand the scope of the 
higher education quality assessment issues addressed in their publications and 
conference presentations to include the areas identified in this research as 
important but sparsely represented in the literature. The importance of the human 
dimensions in assessment found in this research may warrant particular attention by 
those in a position to provide needed skills and understandings in this area. The 
need to attend to this aspect of assessment was largely ignored in the literature 
both as a general concern about practice, and as an area of necessary skills for 
those guiding or conducting assessment on the campuses. 

The ratings of particular standards provided some insight into the current 
status of assessment, and assessment as a process. The standards identified as 
important in the initial planning focused on the need to address the qiaestions of 
what and why to assess within the specific context of each assessment, e.g., 
consideration of the purpose of assessment, and clarification of the objectives or 
mission of the specific institution. This approach supports designing and adapting 
programs to the setting rather than adopting assessment programs from other 
institutions, as was common in early quality assessment activity. 

The inclusion of quantitative information but the absence of qualitative 
information in the list of most important standards for the collection of the data 
may be related to the professional training, orientation, and practice of those 
conducting quality assessment in higher education. The educational background of 
one-thiri of the participants was in organization, finance, planning or 
institutional research, areas which may more likely include the collection and 
analysis of quantitative information, e.g., cost/graduate and F.T.E. Such 
quantifiable data are often the bases for efficiency questions which have dominated 
over questions of effectiveness in higher education in the past. Few of the seventy 
participants identified their educational background as evaluation (n=3) or an area 
within the social sciences (ns?), backgrounds that might include training in the 
collection and analysis of qualitative information. 

The recency of quality assessment activity in higher education, and the 
consequential lack of experience with the utilization of information in the context 
of quality assessment may account for failure of the research to identify standards 
that are very important in the final phase of an assessment, i.e, using the 
findings. This explanation is consistent with an earlier review of literature which 
indicated that standards emphasized by various authors related almost exclusively to 
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the early phases of assessment [19]. The higher proportion of re-ratings found in 
the Phases IV and V may also indicate uncertainty and some of the lack of 
understanding and experience on the part of participants with the later phases. The 
lack of important standards might also indicate that the inventory was incomplete in 
this area, i.e., the two sets of standards from which the inventory was compiled may 
not be a valid representation of applicable standards for utilization in quality 
assessment programs in higher education. The participants were^ however, encouraged 
to add standards they deemed as missing. 

The importance across the assessment phases attributed to standards that 
directly involved people indicated that participants recognized the need to attend 
to the human dimensions of the assessment process. The on*going nature of 
assessment, the ceunpus-wide responsibility and involvement, and the possible 
consequential effects of assessment on individuals may account for the consistently 
important role of the standards that focus on people. 

This on-going nature of assessment and the campus*wide responsibility for its 
conduct may also account for the lack of importance attributed by participants to 
the standard regarding formal obligation in assessment. The continual prccess of 
assessment was also addressed by the two standards that were added to the assessment 
inventory by pilot participants: AS 39 Integration of Data Bases and AS 40 
Longitudinal Reporting. The view of assessment as an on*going process may account 
for the more prominent role of these standards in assessment than in the conduct of 
evaluation in other contexts such as external summative evaluations. 

The research produced a small, but useful subset of standards deemed most 
important by experts for the conduct of quality assessment in higher education. The 
assessment experts considered some standards to be more important than others in the 
conduct of quality assessment* and judged that the standards are more important 
during some phases than at other points in the assessment process. This information 
can be valuable as phase*by*phase guidance for those conducting assessments. 
Although the subset included standards that were identified as of most importance, 
the remaining standards should not be ignored in the conduct of assessment. Because 
of the general lack of experience with assessment, these results may to some extent 
represent uninformed rather than ideal practice. Neverthel^'.s, the findings can 
al&o serve to inform and guide future discussions and research in the sound practice 
of quality assessment in higher education. 
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Assessment Standards (AS) 



AS 1 Kuaan Inter act lone i Those conducting the assessment should respect human 
dignity ana worth in their interactions with other persons associated with an 
assessment f and should show respect for cultural and social values of 
participants. 

AS 2 Described Purposes end Procedures t The purpose of the assessment should 
be clarified; and procedures selected should be appropriate for the purpose. 
Descriptions of the purpose ^ procedures and range of activities should be in 
enough detail^ so that they can be identified and assessed. 

AS 3 Fiscal Responsibility! The allocation and expenditure of assessment 
resources should reflect sound accountability procedures and otherwise be 
prudent and ethically responsible. 

AS 4 Pon&al Obllcratloni What is to be done^ how^ by whom^ and when should be 
agreed to in writing by the formal parties to an assessment. This obligates 
the parties to adhere to all conditions of the agreement or formally 
renegotiate it (includes the release or information) . 

AS 5 Assessment 81de-ef feotsi Those conducting the assessment should bring to 
the attention of decision makers and other relevant audiences suspected side 
effects (positive or negative) of the assessment process itself. 

AS 6 Pull and Prank Disclosure i Oral and written material containing gathered 
information should be open^ direct ^ and honest in the ::losure of pertinent 
findings^ including the limitations of the assessment. 

AS 7 Targeted Dlssealnatloni Persons^ groups^ and organizations who have 
contributed to the assessment should receive feedback appropriate to their 
needs. When appropriate, check draft reports with representatives of 
audiences for clarity and accuracy; consider tailoring dissemination for 
specific audiences using a variety of formats and approaches. 

AS 8 Report Dissemination. Assessment findings should be disseminated to 

clients and other right -to** know audiences » so that they can assess and use the 
findings « 

AS 9 Ivaluator Credlbllltyi The persons conducting the assessment should be 
both trustworthy and conqpetent to perform the assessments 

AS 10 Context Analysis I The context in which the assessment object exists 

should be examined in enough detail to identify the conditions that may affect 
its functioning « 

AS 11 Practical Procedures i The assessment procedures should be practical and 
disruption kept to a mininxun. 

AS 12 Audience Cooperation i The necessary cooperation of those directly 

involved in the assessment as well as other audiences should be planned and 
assurances of cooperation obtained. Efforts should be made to assist 
audiences to develop realistic expectations for assessment and prioritize the 
information requests. 

AS 13 Cost Iffeetlvenessi An estimate of the cost of the proposed assessment 
and, where appropriate^ of alternatives should be provided^ Agreement should 
be reached at the outset that the assessment is likely to produce information 
of sufficient value^ applicability, and potential use to justify its cost. 
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AS 14 * KaoooMndatlonfli Thoa« responsible for making rscoromsndat ions about 
coursss of action, should consider and stats ths rslativs importance of the 
recommendations and the probable effectiveness and costs of the recommended 
courses of action. Recommendations should be clearly distinguished from the 
findings of the assessment on which recommendations were made. 

AS 15 Assessment Impact t Attention should be given to actions that would 
encourage follow-through by members of the audiences. 

AS 16 Information Scope and Selections Information collected should be of such 
scope and selected in such ways as to address pertinent questions about the 
object of the assessment and be responsible to the needs and interest of 
specified audiences. 

AS 17 Report Timeliness t Release of reports should be timely. Be responsible 
when possible I to audiences' changing timetables. 

AS 18 Defensible Information Sources t The sources of information should be 
described in enough detail to assess the adequacy of the information. 
Recognize the conqplementariness of qualitative and quantitative data, and the 
importance of multiple measures. 

AS 19 Political Viability! The assessment should be planned and conducted with 
anticipation of the different positions of various interest groups. 

AS 20 Balanced Reporting of Findings t The assessment should be complete and 
fair in its presentation of the findings, e.g., balanced reporting of data 
that may lead to conclusions about strengths as well as weaknesses, and 
inclusion of intended as well as unintendea outcomes. 

AS 21 Reliable Measurements The information-gathering instruments and 

procedures should be chosen or developed and then implemented in ways that 
will assure that the information obtained is sufficiently reliable for the 
intended use. 

AS 22 Justified Judgments and Recoonendationst Those responsible for making 
judgments and recommendations should explicitly justify such conclusions to 
enable audiences to assess them. Assumptions should be explicitly 
acknowledged. Any reports of such groups should include plausible alternative 
explanations and judgments using the findings, and distinguish among objective 
findings, opinions, judgments, hunches, and speculation. 

AS 23 Rights of Human Subjects! Assessments should be designed and conducted to 
protect and respect the rights and welfare of the human subjects. 

AS 24 Analysis of Quantitative Informationt Quantitative information in an 
assessment should be appropriately and systematically analyzed to ensure 
supportable interpretations. Report potential weaknesses in the data 
collection and describe possible influences on findings. 

AS 25 Object Identifications The purposes and characteristics off the object of 
the assessment to be addressed in the assessment should be specified as 
precisely as possible, e.g., program objectives or institutional mission 
statement . 

AS 26 Public's Right to Xbovt The formal parties to an assessment should 

respect and assure the right to know of various audiences, within the limits 
of other related principles and statutes, such as those dealing with public 
safety and the right to privacy. Restrictions, if any, on access to the data 
and results from an assessment should be clearly established and agreed to at 
the outset • 
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AS 27 rMsibllltyi Thm feasibility of undertaking the assessment should be 
. estimated either informally or through formal analysis and include such 

factors as needed cooper at ion , availability of time, mouityt and expertise, and 
administrative, fiscal, and legal constraints. 

AS 28 Valid Measureaenti The information-gathering instruments and procedures 
should be chosen or developed and then implemented in ways that will assure 
that the interpretation arrived at is valid for the given use. 

AS 29 Responsibility for Uses Those responsible for the assessment should try 
to anticipate and prevent misinterpretations and misuses of the information, 
and when appropriate, make follow-up contacts with users, provide rebuttals of 
misinterpretation, and promote an open exchange of information. Guard against 
parties using collected data for purposes different from those understood by 
persons providing the information. 

AS 30 Conflict of Interests Conflict of interest, frequently unavoidable, 
should be dealt with openly and honestly. 

AS 31 Audience Zdentif icationt Audiences involved in or affected by the 
assessment should be clearly identified. 

AS 32 Ubit of Analysis I The unit of analysis should be appropriate to the 
purpose of the assessment, the types of conclusions to be drawn and the way 
the data are collected. 

AS 33 Assessment Flexibility! Flexibility should be maintained during the 
assessment to respond to changing timetables and other needs. However, 
provision should be made for the detection, reconciliation and documentation 
o^ any departures from the original assessment design, whether unplanned or a 
planned response to needs* 

AS 34 Report Clarityt Reports of results should describe the object being 
assessed and its context, and the purposes, procedures, and findings of the 
assessment. Limitations caused by constraints on time, resources, and data 
availability should also be stated. 

AS 35 Objective Reporting of Results t The assessment procedures should provide 
safeguards to protect the assessment findings and reports against distortion 
by the personal feelings and biases of any party to the assessment « When 
appropriate, draft reports should be checked with representatives of audiences 
for clarity and accuracy. 

AS 36 Valuational Interpretations Those responsible for making judgments about 
quality and recommendations should carefully describe the perspectives, 
procedures, and rationale used to interpret the findings to insure that the 
bases for value judgments are clear. 

AS 37 Data Quality Assurance and Controls The data collected, processed, and 
reported in an assessment should be reviewed and corrected. Analysis of the 
sources of error should be undertaken. 

AS 38 Analysis of Qualitative Informations Qualitative information in an 
assessment should be appropriately and systematically analyzed to ensure 
supportable interpretations. Report potential weaknesses in the data 
collection and describe possible influences on conclusions. 

AS 39 Integration of Data Basess Pertinent, previously collected data that may 
exist in various sources at the institution or that exist outside the 
institution such as available cc\<;iparable data should be utilized and linked to 
any newly collected data. 

AS 40 Longitudinal Report ings Ongoing, update reports of findings should be 
used to keep audiences informed about assessment activity and results. 
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FOOTNOTES 

^ Amarlcan Association of Higher Education (AAHE) , American Evaluation 
Association (AEA) , Association for Institutional Resaarch (AIR), Association 
for the Study of Higher Education (ASHE) , American Educational Research 
Association) AERA, and National Center for Higher Education Management Systems 
(NCHEMS) . 
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TABLE 1 



Number of Standards Rated at Each Level of Importance within 
Each Assessment Phase 



Importance Category 



Very important 
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Assessment Phase 



II III IV 



2 4 2 5 0 



Generally ixi^rtant ^ 5 ^? 

Somewhat important 8 5 4 3 8 
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TABLE 2 

Final Results of Importance Ratings by Phase 
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Assessment Standards in 
General Chronological Order 
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